Move orchestrator config into a model #1295

djeebus · 2025-10-04T00:52:47Z

Pre-requisite for running multiple orchestrators on a single node.

Note

Introduce orchestrator config model and network config, switch port types to uint16 across services, and move Nomad jobs to env-driven ports (drop CLI flags).

Config refactor (orchestrator):
- Add internal/cfg with env-parsed Config (e.g., GRPC_PORT, PROXY_PORT, REDIS_URL|REDIS_CLUSTER_URL, CLICKHOUSE_CONNECTION_STRING, ALLOW_SANDBOX_INTERNET, ORCHESTRATOR_SERVICES, plus NetworkConfig).
- Use cfg.Parse() in main and propagate config (lock path, force stop, services, ports, Redis/ClickHouse, hyperloop port).
- Add tests for config parsing.
Network config:
- Introduce network.Config and ParseConfig; pass through pool/storage/slot; remove internal/consts.go and read hyperloop IP/port from config.
- Update iptables redirect to use per-slot hyperloop port.
Port type unification:
- Change port parameters to uint16 in proxy (shared), orchestrator grpcserver, hyperloop server, sandbox proxy, and client proxy; adjust logs and validation.
Client proxy:
- Validate proxy port range; update NewClientProxy signature and calls.
Jobs (Nomad):
- Add env GRPC_PORT/PROXY_PORT and stop passing --port/--proxy-port args in orchestrator.hcl and template-manager.hcl.
Misc:
- Update benchmarks and build-template command to use network.ParseConfig and uint16 ports.
- Add github.com/caarlos0/env/v11 dependency.

^{Written by Cursor Bugbot for commit 6209e35. This will update automatically on new commits. Configure here.}

packages/orchestrator/main.go

packages/client-proxy/main.go

jakubno

There's also some issue with build in integration tests

packages/orchestrator/internal/sandbox/network/pool.go

djeebus · 2025-10-06T17:14:55Z

The CodeQL issues will be resolved in #1296 , so this is dependent on that one.

# Conflicts: # packages/client-proxy/internal/proxy/proxy.go # packages/client-proxy/main.go

packages/client-proxy/main.go

djeebus · 2025-10-07T22:39:19Z

CodeQL's wrong, %T can't possibly emit sensitive data.

changes were implemented

packages/orchestrator/cmd/build-template/main.go

cursor

Bug: Config Parameter Omission in Slot Creation

The Config parameter, recently added to network.Pool and network.StorageKV, isn't consistently passed to NewSlot calls. This results in network slots being created without essential configuration, such as hyperloop IP and port, leading to compilation errors or runtime issues. This affects both initial slot creation and fallback acquisition.

packages/orchestrator/internal/sandbox/network/pool.go#L81-L88

infra/packages/orchestrator/internal/sandbox/network/pool.go

Lines 81 to 88 in 6209e35

    
           		} 
        
           		zap.L().Info("network slot pool populate closed") 
        
           	}() 
        
           	return pool, nil 
        
           }

packages/orchestrator/internal/sandbox/network/storage_kv.go#L67-L68

infra/packages/orchestrator/internal/sandbox/network/storage_kv.go

Lines 67 to 68 in 6209e35

    
           if status { 
        
           	return NewSlot(key, slotIdx, s.config)

packages/orchestrator/internal/sandbox/network/storage_kv.go#L89-L96

infra/packages/orchestrator/internal/sandbox/network/storage_kv.go

Lines 89 to 96 in 6209e35

    
           if slot == nil { 
        
           	// This is a fallback for the case when all slots are taken. 
        
           	// There is no Consul lock so it's possible that multiple sandboxes will try to acquire the same slot. 
        
           	// In this case, only one of them will succeed and other will try with different slots. 
        
           	reservedKeys, _, keysErr := kv.Keys(s.nodeID+"/", "", nil) 
        
           	if keysErr != nil { 
        
           		return nil, fmt.Errorf("failed to read Consul KV: %w", keysErr)

move orchestrator config into a model

95bfcfb

djeebus requested review from ValentaTomas, dobrac and jakubno as code owners October 4, 2025 00:52

e2b-request-same-site-reviewers bot requested review from tomassrnka and removed request for ValentaTomas, dobrac and jakubno October 4, 2025 00:53

github-advanced-security bot found potential problems Oct 4, 2025

View reviewed changes

packages/orchestrator/main.go Dismissed Show dismissed Hide dismissed

djeebus added 2 commits October 3, 2025 17:58

split network config from the rest

52951ef

fix some more types

775c9d6

This comment was marked as outdated.

Sign in to view

github-advanced-security bot found potential problems Oct 4, 2025

View reviewed changes

packages/client-proxy/main.go Fixed Show fixed Hide fixed

tidy, proof that cursor is wrong

2e58f31

tomassrnka requested a review from jakubno October 6, 2025 07:45

jakubno self-assigned this Oct 6, 2025

jakubno previously requested changes Oct 6, 2025

View reviewed changes

packages/orchestrator/internal/sandbox/network/pool.go Show resolved Hide resolved

parse network config in build-template command

2f0e1f5

This comment was marked as outdated.

Sign in to view

Merge remote-tracking branch 'origin/main' into orchestrator-config

118f424

# Conflicts: # packages/client-proxy/internal/proxy/proxy.go # packages/client-proxy/main.go

github-advanced-security bot found potential problems Oct 6, 2025

View reviewed changes

packages/client-proxy/main.go Fixed Show fixed Hide fixed

proxy port

2c41ca4

This comment was marked as outdated.

Sign in to view

Merge branch 'main' into orchestrator-config

361991b

djeebus requested a review from jakubno October 7, 2025 18:28

This comment was marked as outdated.

Sign in to view

jakubno reviewed Oct 8, 2025

View reviewed changes

packages/orchestrator/cmd/build-template/main.go Outdated Show resolved Hide resolved

Apply suggestion from @jakubno

805f176

jakubno approved these changes Oct 8, 2025

View reviewed changes

Merge remote-tracking branch 'origin/main' into orchestrator-config

3d15ceb

jakubno enabled auto-merge (squash) October 8, 2025 08:34

This comment was marked as outdated.

Sign in to view

jakubno added 2 commits October 8, 2025 11:17

Update test code to the config

8a6dfc9

Move after setting env

6209e35

cursor bot reviewed Oct 8, 2025

View reviewed changes

jakubno merged commit b7ce7cb into main Oct 8, 2025
25 checks passed

jakubno deleted the orchestrator-config branch October 8, 2025 09:30

	}

	zap.L().Info("network slot pool populate closed")
	}()

	return pool, nil
	}


	if slot == nil {
	// This is a fallback for the case when all slots are taken.
	// There is no Consul lock so it's possible that multiple sandboxes will try to acquire the same slot.
	// In this case, only one of them will succeed and other will try with different slots.
	reservedKeys, _, keysErr := kv.Keys(s.nodeID+"/", "", nil)
	if keysErr != nil {
	return nil, fmt.Errorf("failed to read Consul KV: %w", keysErr)

Move orchestrator config into a model #1295

Move orchestrator config into a model #1295

Uh oh!

Conversation

djeebus commented Oct 4, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

jakubno left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

djeebus commented Oct 6, 2025

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

djeebus commented Oct 7, 2025

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Bug: Config Parameter Omission in Slot Creation

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

djeebus commented Oct 4, 2025 •

edited by cursor bot

Loading